Exploration of block-wise dynamic sparseness
نویسندگان
چکیده
Neural networks have achieved state of the art performance across a wide variety machine learning tasks, often with large and computation-heavy models. Inducing sparseness as way to reduce memory computation footprint these models has seen significant research attention in recent years. In this paper, we present new method for dynamic sparseness, whereby part computations are omitted dynamically, based on input. For efficiency, combined idea block-wise matrix-vector multiplications. contrast static which permanently zeroes out selected positions weight matrices, our preserves full network capabilities by potentially accessing any trained weights. Yet, matrix vector multiplications accelerated omitting pre-defined fraction blocks from matrix, Experimental results task language modeling, using recurrent quasi-recurrent models, show that proposed can outperform baselines. addition, reach similar modeling perplexities dense baseline, at half computational cost inference time.
منابع مشابه
A Block-Wise random sampling approach: Compressed sensing problem
The focus of this paper is to consider the compressed sensing problem. It is stated that the compressed sensing theory, under certain conditions, helps relax the Nyquist sampling theory and takes smaller samples. One of the important tasks in this theory is to carefully design measurement matrix (sampling operator). Most existing methods in the literature attempt to optimize a randomly initiali...
متن کاملBlock-Wise Non-Malleable Codes
Non-malleable codes, introduced by Dziembowski, Pietrzak, and Wichs (ICS ’10) provide the guarantee that if a codeword c of a message m, is modified by a tampering function f to c′, then c′ either decodes to m or to “something unrelated” to m. It is known that non-malleable codes cannot exist for the class of all tampering functions and hence a lot of work has focused on explicitly constructing...
متن کاملdynamic coloring of graph
در این پایان نامه رنگ آمیزی دینامیکی یک گراف را بیان و مطالعه می کنیم. یک –kرنگ آمیزی سره ی رأسی گراف g را رنگ آمیزی دینامیکی می نامند اگر در همسایه های هر رأس v?v(g) با درجه ی حداقل 2، حداقل 2 رنگ متفاوت ظاهر شوند. کوچکترین عدد صحیح k، به طوری که g دارای –kرنگ آمیزی دینامیکی باشد را عدد رنگی دینامیکی g می نامند و آنرا با نماد ?_2 (g) نمایش می دهند. مونت گمری حدس زده است که تمام گراف های منتظم ...
15 صفحه اولذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Pattern Recognition Letters
سال: 2021
ISSN: ['1872-7344', '0167-8655']
DOI: https://doi.org/10.1016/j.patrec.2021.08.013